Extended depth of focus for high-resolution optical image scanning

ABSTRACT

Methods and systems for acquiring and/or projecting images from and/or to a target area are provided. Such a method or system can includes an optical fiber assembly which may be driven to scan the target area in a scan pattern. The optical fiber assembly may provide multiple effective light sources (e.g., via a plurality of optical fibers) that are axially staggered with respect to an optical system located between the optical fiber and the target area. The optical system may be operable to focus and/or redirect the light from the multiple light sources onto separate focal planes. A composite image may be generated based on light reflected from and/or projected onto the separate focal planes. The composite image may have an extended depth of focus or field spanning over a distance between the separate focal planes while maintaining or improving image resolution.

CROSS-REFERENCE

This application is a U.S. National Phase Application under 35 U.S.C.371 of International Application No. PCT/US13/26527, filed Feb. 15,2013, which claims the benefit of U.S. Provisional application No.61/599,839, filed Feb. 16, 2012, which applications are incorporatedherein by reference.

STATEMENT AS TO FEDERALLY SPONSORED RESEARCH

This invention was made with government support under Grant No. R33CA094303 awarded by the National Institutes of Health. The governmenthas certain rights in the invention.

BACKGROUND OF THE INVENTION

In an image scanning system, a beam of light is typically scanned acrossa target area and the backscattered or reflected light is modulated inintensity and color to give a pixel at the particular location and timewithin the scan corresponding to the point on the surface. To increaseimage resolution or decrease the size of the image pixel, the opticalpoint of interrogation on the surface need to be made sufficientlysmall. However, when the light is focused to a small spot, the depth offocus or depth of field of the resulting image becomes relativelyshallow as well, meaning that only a small portion of the target arearemain in focus. There is a need, therefore, to provide an extendeddepth of focus or field while maintain high image resolution.

SUMMARY OF THE INVENTION

The following presents a simplified summary of some embodiments of theinvention in order to provide a basic understanding of the invention.This summary is not an extensive overview of the invention. It is notintended to identify key/critical elements of the invention or todelineate the scope of the invention. Its sole purpose is to presentsome embodiments of the invention in a simplified form as a prelude tothe more detailed description that is presented later.

In many embodiments, improved methods and systems for acquiring and/orprojecting images from and/or to a target area are provided. Suchmethods and systems include an optical fiber assembly which may bedriven to scan the target area in a scan pattern. The optical fiberassembly may provide multiple effective light sources that are axiallystaggered with respect to an optical system that may focus and/orredirect the light from the multiple light sources onto separate focalplanes and/or into optical beams having different degrees ofcollimation. Image data corresponding to the separate focal planes maybe combined to generate a composite image with an extended depth offocus or field and an improved image resolution.

In one aspect, a method for acquiring an image of a target area isprovided. The method includes scanning an optical fiber assembly in apattern so as to provide illumination to the target area, the opticalfiber assembly comprising a plurality of optical fibers opticallycoupled to an optical system to focus light from the optical fiberassembly and produce a plurality of focal planes located at differentdistances with respect to an optical axis in the optical system,collecting, for each of the plurality of focal planes, light reflectedfrom the target area, and generating, based at least in part on thelight collected from the target area, a composite image having a depthof field spanning over a distance between the plurality of focal planes.

In one aspect, a system for acquiring images of a target area isprovided. The system includes an optical fiber assembly comprising aplurality of optical fibers operable to provide illumination to thetarget area, an optical system optically coupled with the optical fiberassembly, the optical system operable to focus illumination from opticalfiber assembly to produce a plurality of focal planes located atdifferent distances with respect to an optical axis in the opticalsystem, a scanner actuator coupled to the optical fiber assemblyconfigured to actuate distal portions associated with the plurality ofoptical fibers to scan, either together or individually, in a scanpattern on the plurality of focal planes, and one or more detectorsconfigured to detect, for each of the plurality of focal planes, lightreflected from the target area. The system also includes a processorcomprising a tangible medium, the tangible medium comprisinginstructions that when executed cause the processor to generate, basedat least in part on the light collected from the target area, acomposite image having a depth of field spanning over a distance betweenthe plurality of focal planes.

In one aspect, a method for projecting a composite image to a targetarea is provided. The method includes scanning an optical fiber assemblyin a scan pattern so as to provide illumination to the target area, theoptical fiber assembly comprising a plurality of optical fibersoptically coupled to an optical system to focus light from the opticalfiber assembly and produce a plurality of focal planes located atdifferent distances with respect to an optical axis in the opticalsystem, modulating the intensity of the provided light to createvariation in light intensity across the target area, and projecting,based at least in part on the illumination provided to the target area,a composite image onto the target area, the composite image having adepth of field spanning over a distance between the plurality of focalplanes.

In one aspect, a system for projecting images onto a target area isprovided. The system comprises an optical fiber assembly comprising aplurality of optical fibers operable to provide illumination to thetarget area, an optical system optically coupled with the optical fiberassembly, the optical system operable to focus illumination from opticalfiber assembly to produce a plurality of focal planes located atdifferent distances with respect to an optical axis in the opticalsystem, a scanner actuator coupled to the optical fiber assemblyconfigured to actuate distal portions associated with the plurality ofoptical fibers to scan, either together or individually, in a scanpattern on the plurality of focal planes, and one or more processorscomprising a tangible medium, the tangible medium comprisinginstructions that when executed cause the one or more processors toproject, based at least in part on the illumination provided to thetarget area, a composite image having a depth of field spanning over adistance between the plurality of focal planes.

INCORPORATION BY REFERENCE

All publications, patents, and patent applications mentioned in thisspecification are herein incorporated by reference to the same extent asif each individual publication, patent, or patent application wasspecifically and individually indicated to be incorporated by reference.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth with particularity inthe appended claims. A better understanding of the features andadvantages of the present invention will be obtained by reference to thefollowing detailed description that sets forth illustrative embodiments,in which the principles of the invention are utilized, and theaccompanying drawings of which:

FIG. 1 illustrates components of an example image scanning system forimaging a target area, in accordance with an embodiment.

FIG. 2 illustrates an example scanning device that may be used in theimage scanning system, in accordance with an embodiment.

FIGS. 3a-b illustrate example components of a scanning device configuredto provide an extended depth of field, in accordance with an embodiment.

FIG. 4 illustrates example image processing used to provide an imagewith an extended depth of focus or depth of field and improvedresolution, in accordance with an embodiment.

FIG. 5 illustrates an example process for acquiring an image of a targetarea, in accordance with an embodiment.

FIG. 6 illustrates an example process for generating a composite imageof a target area with an extended depth of focus or depth of field, inaccordance with an embodiment.

FIG. 7 illustrates some example configurations of optical fibers in anoptical fiber assembly, in accordance with some embodiments.

FIG. 8 illustrates some example optical fibers with angled end faces inan optical fiber assembly, in accordance with some embodiments.

FIGS. 9a-c illustrate an example optical fiber assembly, in accordancewith an embodiment.

FIGS. 10a-d illustrate some example mechanical cantilever assemblies, inaccordance with some embodiments.

FIG. 11 illustrates an example process for projecting a composite imageonto an area, in accordance with embodiments.

DETAILED DESCRIPTION OF THE INVENTION

The present invention provides techniques for creating an extended depthof focus or field while maintaining high image resolution in an imagescanning system, such as a scanning fiber endoscope (SFE). Typically, anSFE has a single mode optical fiber that is driven to scan in a scanpattern (e.g., spiral or zigzagged) to deliver illumination in aspace-filling area over the surface to be imaged. An optical system(e.g., an objective lens system) between the scanning optical fiber tipand the target area typically defines the image resolution in the SFEand the optical system is typically designed to achieve high spatialresolution while providing shallow depths of focus or field.

According to an embodiment of the present invention, the single scanningoptical fiber in an SFE is replaced with an optical-fiber assemblycomprising a plurality of optical fibers. The tips of these opticalfibers may be axially staggered relative to an optical system that isconfigured to focus light from the optical fibers onto the target area.The staggered fiber tips may be operable to provide varying light sourcepoints which may be focused, by the optical system, into separate orstaggered focal points and hence separate focal planes. Each focal planemay be associated with a very limited depth of focus or field of view,but in combination the resulting images of the target area may have adesirable extended depth of focus or field with improved imageresolution.

FIG. 1 illustrates components of an example image scanning system 100for imaging a target area, in accordance with an embodiment. In someembodiments, the image scanning system 100 may be implemented by anendoscope, bronchoscope, colonoscope or any other type of scopes orinstrument to image the surfaces of tissues or organs such as frominside of a body cavity or lumen. More generally, the image scanningsystem 100 may be used to image any types of surfaces.

In an embodiment, the image scanning system 100 includes a base station102 and a scanning device 104. The scanning device 104 may be coupled tothe base station 102 electronically and/or optically, for example, viaone or more cables (not shown). In various embodiments, the imagescanning system 100 may include more or less components than those shownin FIG. 1.

The base station 102 may include a processing and display system 106, anoptical scanner driver 108, a detector 110, output/input optics 112, alight source driver 114, and a light source 116. In some embodiments,one or more of these elements are omitted. For example, in an embodimentfor a fiber scanned display system, the detector 110 is omitted. Inanother embodiment, the light source driver 114 is omitted.

The processing and display system 106 may include one or more processingunits (collectively referred to as the “controller”) and a memory (notshown). The memory may comprise a random access memory (“RAM”), a readonly memory (“ROM”), and/or a permanent storage device, such as a floppydisc, tape drive, disk drive, DVD/CD-ROM drive, memory card, USB flashdrive, solid state drive (SSD) and the like.

In various embodiments, the memory may be coupled to or separate fromthe controller. The memory may be used for storing software modules,look-up tables, and algorithms that control the operation and anycalibration of the image scanning system 100. In some embodiments, acontrol routine stored in the memory may be used by the processor tocontrol the optical scanner assembly 118 and light source 116. Thecontrol routine may be configurable so as to provide operatingparameters of the optical scanner assembly 118 (e.g., resonantfrequency, voltage limits, zoom capability, color capability, etc.). Thememory may also be used for storing data (e.g., images, parameters)received or produced by the processing and display system 106. Forexample, such data may concern reflection from the target area, imagingdata associated with the target area, and the like.

The controller may include one or more microprocessors and/or one ormore dedicated electronics circuits, which may include a gate array (notshown). The controller may also include scanner drive electronics,sensor amplifiers and A/D converters (not shown). In variousembodiments, the controller may be capable of executing executableinstructions or control routines stored in the memory to control variousaspects of the image scanning system 100. For example, the controllermay be capable of generate, process and/or cause display of images basedat least in part on light detected by the detector 110. In anembodiment, the controller also controls actuation of the opticalscanner assembly 118 by sending electrical control signals to theoptical scanner driver 108. In another embodiment, the controllerconsists of field programmable gate arrays (FPGA) which are used tocondition the fiber scanning pattern and to map the scanned lightcollection into video image files and display frames. In someembodiments, the controller may also control (e.g., activate) the lightsource 116 by sending electrical control signals to the light sourcedriver 114. As will be appreciated by those skilled in the art, themethods and techniques of the present invention may be carried out bysoftware modules executed by the controller and/or by electronichardware.

In some embodiments, the controller is in communication with otherelements of the image scanning system 100 via output/input electronics(not shown). The output/input electronics may allow for electricalcommunication between the controller and the other elements of the imagescanning system 100 by providing appropriate interfaces as known in theart.

The processing and display system 106 may optionally include a display(not shown). The display may be used to display images acquired by theimage scanning system 100, analysis results and/or other data. Forexample, the display may be configured to provide real-time images of apatient tissues or organs as they are being scanned, indication of thehealth or condition of scanned areas and the like. In addition, thedisplay may provide a graphical user interface (GUI) to a user operatingthe image scanning system. For another example, the display may be usedto receive and execute user commands.

In an embodiment, the base station 102 also includes a light source 116to provide light to the scanning device 104 through one or moreoutput/input optics 112. The light source 116 may include any lightsource suitable for image acquisition. Examples of suitable lightsources include, but are not limited to, lasers, laser diodes, verticalcavity surface-emitting lasers (VCSELs), light-emitting diodes (LEDs),other light emitting devices known in the arts, and combinationsthereof. In various example embodiments of the invention, the lightsource may include a red light source, a blue light source, a greenlight source, an RGB light source, a white light source, an infraredlight source, an ultraviolet light source, a high intensity therapeuticlaser light source, or a combination thereof. Depending on theparticular implementation, the light source may emit a continuous streamof light, modulated light, or a stream of light pulses. The lightsources may be configured to be switchable between a first mode (e.g.,continuous stream) and a second mode (e.g., stream of light pulses). Ifa plurality of light sources are used, a combiner may be used.

In an embodiment, the base station 102 also optionally includes a lightsource driver 114 configured to communicate electrical control signalsto the light source 116 for activating, deactivating or otherwisecontrolling the one or more light sources. The light source driver 114may be controlled by a controller of the processing and display system106 as discussed above.

In an embodiment, the base station 102 also optionally includes anoptical scanner driver 108 that communicates electrical drive signals tothe optical scanner 18 for actuating cantilevered distal portion of theoptical scanner assembly 118, for example, according to an electricalcontrol signal received from a controller of the processing and displaysystem 106 as discussed above. In some embodiments, the drive signalsinclude two drive signals for actuating the optical fiber assembly in ascan pattern about two separate axes. The optical scanner driver 108 maybe implemented in hardware (e.g., a circuit), software (e.g., a routine,program, or set of machine-executable instructions), or a combination ofboth.

In an embodiment, the base station 102 also optionally includes adetector 110, such as a photo detector, for detecting and/or measuringlight reflected from the target area. Examples of the detector 110 mayinclude photodiodes, photomultiplier tubes (PMTs) and the like. In manyembodiments, the detector 110 converts the reflected light into anelectrical signal and communicates the electrical signal to theprocessing and display system for further processing (e.g., imagegeneration) and/or display.

In some embodiments, the detector 110 may not receive light communicatedvia the output/input optics 112. Rather, the detector 110 may be placedanywhere where it can detect, and/or optionally measure, light reflectedfrom the target area. For example, in an embodiment, the detector 110 islocated inside the optical scanner assembly 118 so as to receivereflected light communicated only through the optical system 120. Inanother embodiment, the detector 110 is located outside to the opticalscanner assembly 118.

In various embodiments, the output/input optics 112 may be used forcommunicating light from the light source to the optical scannerassembly 118 and/or for communicating reflected light to the detector110. In various embodiments, the output/input optics 112 may include awaveguide (e.g., optical fiber), a lens assembly or other means forcommunicating light as is known in the art. In an embodiment, at least aportion of the output/input optics 112 is enclosed in one or more cablesconnecting the base station 102 and the scanning device 104.

Refer now to the scanning device 104. Typically, the scanning device 104may be relatively small and maneuverable compared to the base station102. In some cases, the scanning device 104 may be suitable to beinserted into a body cavity or lumen. In an embodiment, the scannerdevice 104 includes an optical scanner assembly 118 and optionally anoptical system 120. In an embodiment, the optical scanner assembly 118includes a scanner actuator that is operable to actuate a cantilevereddistal portion of a waveguide assembly according to drive signals fromthe optical scanner driver 108. The drive signals may include two drivesignals for actuating the cantilevered distal portion of the waveguideassembly in a scan pattern about two separate axes. In an embodiment,the waveguide assembly includes an optical fiber assembly comprising oneor more optical fibers.

In some other embodiments, the optical scanner assembly 118 may includea mirror or other reflective device that may be moved by an actuator toscan a reflected beam. In yet some other embodiments, the opticalscanner assembly 118 may include a lens or other focusing device thatmay be moved by an actuator to scan a focused beam.

The scanning device 104 may optionally include an optical system 120comprising a lens assembly for directing and focusing light directed outof the optical scanner assembly 118 onto the target area 101. In someembodiments, the optical system 120 may also direct and/or focus lightreflected from the target area to the optical scanner assembly 118and/or to the detector 110.

In some embodiments, the scanning device 104 may include multipleelements or systems such as discussed above (e.g., actuators,waveguides, optical systems scanners and the like) that collectivelyperform one or more of the functions discussed above.

In some embodiments, the scanning device may implement technologiesdeveloped for a scanning fiber endoscope (SFE) as described in numerouscommonly owned U.S. patent applications and patents, which, for example,include: U.S. Pat. No. 7,298,938, entitled “Configuration Memory for aScanning Beam Device”, filed on Oct. 1, 2004; U.S. patent applicationSer. No. 10/956,241, entitled “Remapping Methods to Reduce Distortionsin Images,” filed on Oct. 1, 2004; U.S. Pat. No. 7,159,782, entitled“Methods of Driving a Scanning Beam Device to Achieve High Frame Rates,”filed on Dec. 23, 2004; U.S. Pat. No. 7,784,697, entitled “Methods ofDriving a Scanning Beam Device to Achieve High Frame Rates,” filed onJan. 3, 2008; U.S. Pat. No. 7,189,961, entitled “Scanning Beam Devicewith Detector Assembly,” filed on Feb. 23, 2005; U.S. patent applicationSer. No. 11/094,017, entitled “Methods and Systems for CreatingSequential Color Images,” filed on Mar. 29, 2005; U.S. Pat. No.7,312,879, entitled “Distance Determination in a Scanned Beam ImageCapture Device,” filed on Aug. 23, 2005; U.S. Pat. No. 7,395,967,entitled “Methods and Systems for Counterbalancing a Scanning BeamDevice,” filed on Jul. 21, 2005; and U.S. patent application Ser. No.12/040,249, entitled “Piezoelectric Substrate Fabrication and RelatedMethods,” filed on Feb. 29, 2008; the complete disclosures of which areincorporated herein by reference.

FIG. 2 illustrates an example scanning device 200 that may be used in animage scanning system, in accordance with an embodiment. For example,the scanning device 200 may be used in an image scanning system 100 suchas discussed in connection with FIG. 1.

In this example, the scanner device 200 includes a cantilevered distalportion 207 of a waveguide assembly 209, with a distal tip 208, mountedwithin a housing 206. The waveguide assembly 209 may include an opticalfiber assembly comprising one or more optical fibers. In otherembodiments, the waveguide assembly 209 may include one or morenon-optical fiber waveguides. In an embodiment, the waveguide assembly209 is fixed along at least one point of the waveguide assembly 209 soas to be cantilevered such that the distal portion 207 is free to bedeflected. In some embodiments, the cantilevered distal portions of someof the waveguides in the waveguide assembly 209 may be coupled or boundtogether in a suitable configuration (e.g., side by side) to form acombined cantilevered distal portion. In other embodiments, some or allof the waveguides may not be bound together.

In various embodiments, the cantilevered distal portion 207 of thewaveguide assembly 209 may have any desired dimensions and/orcross-sectional profile. For example, the distal portion 207 can have asymmetrical cross-sectional profile or an asymmetrical cross-sectionalprofile, depending on the desired characteristics of the scanningdevice. A distal portion 207 with a round cross-sectional profile willtypically have substantially the same resonance characteristics aboutany two orthogonal axes, while a distal portion 207 with an asymmetriccross-sectional profile (e.g., an ellipse) will typically have differentresonant frequencies about the major and minor axes. In someembodiments, the distal portion 207 may be linearly or non-linearlytapered along its length.

In an embodiment, the waveguides comprising the waveguide assembly 209are arranged such that the distal tips of the waveguides are staggeredby a small axial distance (e.g., less than 1 to 2 mm) with respective toan optical axis of the optical system 214 (e.g., an objective lensassembly). Thus, when the light coming out of the distal tips of thewaveguide assembly 209 is redirected and/or focused by the opticalsystem 214, separate and axially shifted or staggered illuminationplanes and/or focal planes are created relative to the optical axis ofthe optical system 214.

In an embodiment, the distal portion 207 of the waveguide assembly 209is coupled to and actuated by a scanner actuator 210. Examples ofsuitable types of scanner actuators include, but are not limited topiezoelectric tubes, Electroactive Polymer (EAP) tubes, other actuatortubes, other piezoelectric actuators, other EAP actuators, magneticactuators, electromagnetic actuators, electrostatic actuators, sonicactuators, electroacoustic actuators, electromechanical actuators,microelectromechanical systems (MEMS), and other transducer capable ofmoving the cantilevered distal portion of a waveguide assembly.

In an embodiment, an optical scanner driver such as described inconnection with FIG. 1 may supply drive signals to the scanner actuator210 to actuate the distal portion 207 in a scan pattern. The opticalscanner driver may be provided inside or outside of the housing 206. Thescan pattern may be one dimensional (e.g., zigzag lines) ormulti-dimensional (e.g., spiral pattern). In an embodiment, the distalportion 207 of the waveguide assembly 209 is driven at a frequency thatis within a Q-factor of the resonant frequency of the distal portion207, and preferably at its mechanical or vibratory resonant frequency(or harmonics of the resonant frequency). As may be appreciated, thedistal portion 207 does not have to be driven at substantially theresonant frequency.

In an embodiment, a housing 206 surrounds the distal portion 207 and thescanner actuator 210. The scanner actuator 210 may be mounted within thehousing 206 via one or more collars 212. The housing 206 can also houseall or a portion of the optical system 214. The optical system 214 maybe spaced from the distal end 208 of the distal portion 207 so as tofocus and/or redirect light emitted from the distal end 208 to a targetarea. In some embodiments, the optical system 214 is provided tofacilitate improved image resolution and/or field of view for theoptical scanner assembly. In some embodiments, the optical system 214 orone or more components (e.g., lens) thereof may be fixed relative to thedistal end 208 of the distal portion 207 and/or relative to the scanneractuator 210. In other embodiments, the optical system 214 or one ormore components (e.g., lens) thereof may be movable relative to thehousing 206.

In an embodiment, the scanning device 200 comprises an outer sheathing202 outside the scanner housing 206 that includes one or more returnwaveguides 204 (e.g., optical fibers). The return waveguides 204 may beconfigured to carry light reflected from a target area back to a basestation for further processing and analysis (e.g., by light detectors,processors and the like). In some embodiments, light detectors may belocated so as to receive reflected light via the optical fiber. In someembodiments, the detectors may be located outside the optical path ofthe waveguide assembly 209. For example, the detectors may be located ina base station or near the distal portion of scanner assembly butoutside the housing 206. In other embodiments, the detectors may belocated inside the housing 206.

When in use, the waveguide assembly 209 may be driven by scanneractuator 210 to scan, in a scan pattern (e.g., spiral, zigzag), onto oneor more illumination planes 218 that are proximal to the distal end ofthe scanner assembly. Each waveguide of the waveguide assembly 209 may,through the optical system 214, generate an illumination spot on anillumination plane 218. The illumination spot may be located at or neara focal point to enhance image resolution. As such, the illuminationplane may coincide with or may be very close to the focal planeassociated with the focal point. When the distal ends of the waveguidesof the waveguide assembly 209 are axially staggered as described above,separate focal points and hence focal planes associated with the focalpoints may be created by a single optical system 214. Light reflectedfrom each of these focal planes may be collected and processed togenerate an image of extended depth of focus or field deeper than thedepth of focus or field associated with any single focal plane.

FIG. 3a illustrates example components of a scanning device 300configured to provide an extended depth of field, in accordance with anembodiment. The scanning device 300 includes an optical scanner assembly301 and an optical system 302. The optical scanner assembly 301 includesa scanner actuator 304 and an optical fiber assembly 303. The scanneractuator 304 may be coupled to the optical fibers assembly 303 so as toactuate a cantilevered distal portion of the optical fibers assembly 303in a scan pattern, as discussed in connection with FIG. 2.

In an embodiment, the optical fiber assembly 303 includes cantileveredportions of a plurality of optical fibers such as 308, 310, 312 and 314.The number, dimensions and material of the optical fibers in the opticalfiber assembly 301 may vary in different embodiments, for example, basedat least in part on the application of the scanning device. In anembodiment, three or four single mode optical fibers made from silicondioxide (quartz) material have their claddings etched from the standard125 microns in outer diameter to approximately 10 to 50 microns diameterin outer diameter and the lengths of the cantilevered portions of theoptical fibers are about 2 to 8 mm proximally. Note that in thisexample, the etching of the claddings is not required; rather, theetching procedure is used simply to save space for microendoscopyapplication so as to create a multi-waveguide cantilever ofapproximately the same overall diameter as a 125 micro single opticalfiber used in a micro-optical fiber scanner.

In an embodiment, as shown in FIG. 3a , the distal portions of theoptical fibers in the optical fiber assembly may be arranged in astaggered fashion along an optical axis 305 of the optical system 302such that distal tips of the optical fibers are of varying distances tothe lens of the optical system 302. For example, as shown in FIG. 3a ,the distal tip of optical fiber 310 may be staggered by a small axialdistance 307 (e.g., 1 to 2 mm) from the distal tip of adjacent opticalfiber 308. Similarly, the distal ends of optical fibers 312 and 314 maybe staggered by a small axial distance from the distal ends of opticalfibers 310 and 312, respectively. In various embodiments, the opticalfibers may be arranged in various configurations. For example, thestaggered distance between adjacent optical fibers need not be uniform.Some of the adjacent optical fibers may not be staggered at all. In someembodiments, the optical fibers may be arranged side by side as shown inFIG. 3a . In other embodiments, the optical fibers may be arranged alongmultiple axes. Some example configurations of the optical fibers areprovided in FIG. 7.

In some embodiments, the cantilevered portions of the optical fibers inthe optical fiber assembly may be bound or coupled together to form acombined cantilevered portion. The combined cantilevered portion may beetched or tapered to reduce the overall diameter. Various techniques maybe used to bind the optical fibers including binding or welding of theoptical fibers using carbon dioxide laser, heat, optical epoxy, adhesivechemical, heat-shrunk physical tubing and the like. Such fusedcantilevered portion of the optical fibers may be driven by the scanneractuator as a whole in a scan pattern similar to the way a singleoptical fiber is driven to scan in a scanning fiber endoscope (SFE). Insome other embodiments, at least some of the cantilevered portion of theoptical fibers in the optical fiber assembly may not be bound together.Instead, some of the cantilevered portions may be free and separate fromone another. For example, in an embodiment, the optical fibers may bedivided into multiple sub-groups each of which is bound together. Inanother embodiment, all the optical fibers may be free from each other.In such embodiments, the optical fibers may be actuated, sequentially orsimultaneously, by one or more scanner actuators.

In an embodiment, light emitted from the staggered distal tips of theoptical fibers 308, 310, 312, and 314 is focused and/or redirected by anoptical system 302 to produce corresponding focal points or illuminationspots 316, 318, 320 and 322 that are also staggered along the opticalaxis 305. Typically, the closer the light source is to the lens of theoptical system, the further the focal point is along the optical axisfrom the optical system and vice versa. The focal points 316, 318, 320and 322 are associated with corresponding focal plane 324, 326, 328 and330. As used herein, a focal plane is a plane (shown as the plane of Xand Y axis in FIG. 3a ) that passes through a focal point and isperpendicular to an optical axis (shown as the Z axis). As shown, focalplanes 324, 326, 328 and 330 are separately spaced along the opticalaxis each with a corresponding depth of field 332, 334, 336 and 338.Typically, each of these depths of field is intended to be shallow inexchange for a relatively small illumination spot and hence improvedimage resolution. However, when light reflected from each of theplurality of focal planes collected and processed, a composite image maybe generated, based at least in part on imaging data from the multiplefocal planes, with a combined depth of the field 340 that spans at leastover the distance over the multiple focal planes. Such an extended depthof field 340 may be deeper than the depth of field associated with anyof the individual focal planes.

In a preferred embodiment, the optical system 302 remains substantiallyfixed relative to the optical fiber assembly and/or actuator during thescanning process. Alternatively, the optical system may be moved (e.g.,via moving mirrors or lens) to produce a scanning light beam.Advantageously, when the optical system is fixed, the optical fiberassembly may be driven to scan at a scan rate and/or pattern independentof the mechanical limits of the optical system.

It is understood that the focal points and focal planes illustrated inFIG. 3a represents all the focal points and focal planes that may occurover time. In some embodiments, the staggered optical fibers may beconfigured to provide illumination in a sequential manner such that notall focal planes or all portions of the focal planes are illuminated atonce. In some embodiments, the sampling of the collected light may beperformed at a frame-sequential, line-sequential or evenpixel-sequential manner.

In some embodiments, the optical system may focus and/or redirect lightfrom the optical fiber assembly into light beams with different degreesof collimation. FIG. 3b illustrates such an example, in accordance withanother embodiment. In this embodiment, the optical fiber assembly 350includes a plurality of optical fibers 356, 358 and 360 that arearranged such that the distal ends of the optical fibers have staggeredends with different distances 357, 359 and 361, respectively, withrespect to an optical system 352. For example, the light emerging fromthe distal end of the optical fiber waveguide 358 is located at thedistance 359 which is equivalent to the focal length of the lens system352, which results in a more collimated beam of light compared to theother optical fiber tips located closer to the lens system. During thescanning process, the optical fiber assembly 350 may be driven to movein a vibratory motion. As illustrated, due to the different focal lengthand/or the configuration of the optical system, light beams from thelight sources may become mostly collimated (e.g., for optical fiber 358)or divergent (e.g., for optical fiber 356). This embodiment is mostrelevant to a system that is generating images directly to the eye for afiber scanning display system that has the capability for displayingimages at various depths. The image being scanned with a beam ofcollimated light, the eye would relax the crystalline lens of the eye tobring the apparent distant image into focus. Whereas, the beams ofdiverging light would appear out of focus until the eye accommodates tobring the apparent nearer images into focus. In this way a singlemicrodisplay based at least in part on scanning multiple channels oflight at different degrees of collimation can generate a truethree-dimensional display.

As discussed above, each focal or illumination plane associated witheach of the plurality of optical fibers may have a very limited depth offocus or field of view. However, in combination, the resulting images ofthe target area may have the desired extended depth of field at a highresolution. FIG. 4 illustrates example image processing to provide anextended depth of field, in accordance with an embodiment. In thisexample, light reflected from each of the plurality of illumination orfocal planes generated by staggered-tipped optical fiber assembly may becollected (simultaneously or separately) and processed to generate aplurality of image data sets for a target area. Each image set of theplurality of image data sets may correspond to an image for one of theplurality of focal planes. For example images data sets 402, 404, 406and 408 may be generated for focal planes 332, 334, 336 and 338,respectively, discussed in connection with FIG. 3a . Since each focalplane is associated with a limited depth of focus or depth of field,only a small portion of corresponding image (indicated by shaded areasin FIG. 4) may have a high resolution relative to the rest of the image.In an embodiment, each image data set may be processed or analyzed, forexample, by an image processing system 106 discussed in connection withFIG. 1, to identify the regions or pixels within the image with thehighest focus or resolution. In an embodiment, such identifiedhigh-resolution regions or pixels across multiple images may be combinedinto a composite image 410 of the target area. Thus, the composite image410 has an effective depth of field that extends beyond the depths offield associated with the individual focal planes.

FIG. 5 illustrates an example process 500 for acquiring an image of atarget area, in accordance with embodiments. Aspects of the process 500may be performed, for example, by the image scanning system 100discussed in connection with FIG. 1. Some or all of the process 500 (orany other processes described herein, or variations and/or combinationsthereof) may be performed under the control of one or morecomputer/control systems configured with executable instructions and maybe implemented as code (e.g., executable instructions, one or morecomputer programs or one or more applications) executing collectively onone or more processors, by hardware or combinations thereof. The codemay be stored on a computer-readable storage medium, for example, in theform of a computer program comprising a plurality of instructionsexecutable by one or more processors. The computer-readable storagemedium may be non-transitory. The order in which the operations aredescribed is not intended to be construed as a limitation, and anynumber of the described operations may be combined in any order and/orin parallel to implement the processes.

In an embodiment, process 500 includes scanning 502 the target area togenerate a plurality of distinct focal planes. In an embodiment, anoptical fiber assembly comprising a plurality of optical fibers may beactuated to scan in a scan pattern on the target area through an opticalsystem, such as discussed in connection with FIG. 3a . The distal tipsof the plurality the optical fibers may be axially staggered relative toan optical axis of the optical system such that the resulting focalpoints and hence focal planes are also axially staggered or spacedapart. In various embodiments, the staggered optical fibers may beconfigured to illuminate the respective scanning planes, sequentially orsimultaneously, individually or as a whole.

Light reflected from the target area may be collected 504, for example,by one or more lens and/or light-collection optical fibers and providedto one or more detectors such as discussed in connection with FIG. 1.For example, the light may be emitted and detected from the same opticalfiber in a confocal arrangement. In various embodiments, lightcollection and/or detection may be performed in a frame-sequential,scan-line-sequential pixel-sequential or any suitable manner. In anembodiment, the collection fibers and/or detectors may be divided intotwo or more groups based at least in part on the relative positionand/or orientation of the fibers tips. For example, the fibers may bedivided into four quadrants based at least in part on their positionsrelative to the scanning fiber tips. In an embodiment, the collectionfibers and/or detectors may be divided into two or more groups accordingto the optical fibers in the optical fiber assembly.

In an embodiment, each of the plurality of illumination or focal planesis scanned in sequence and the backscattered light associated with eachof the illumination or focal planes is collected and detected insequence. In another example, two or more of the plurality ofillumination or focal planes are scanned at a time and the backscatteredlight associated with the two or more illumination or focal planes iscollected and detected at a time. When the planes are scanned insequence, the same set of detectors may be used to detect lightassociated with different illumination planes. In yet anotherembodiment, all of the illumination or focal planes are scannedsimultaneously and the backscattered light associated all of the planesis collected at once.

Based at least in part on the detected backscattered light reflectedfrom the target area, a composite image having a depth of field spanningover a distance between the plurality of focal planes may be generated506. To generate the composite image, image data associated with each ofthe focal planes may be processed and/or combined. For example, for agiven pixel or a set of pixels in a particular region of the compositeimage, the image data associated with various focal planes may beanalyzed to extract the image data with the highest resolution. In otherwords, knowing beforehand where the focal depth is for each of theplurality of optical fibers, the regions or pixels of highest focus maybe combined in a three-dimensional mapping to the correct X and Y pixellocation of the image and the Z location of each of the plurality offocal planes. In other embodiments, other suitable image processingtechniques may also be used to otherwise combine the image data. Forexample, for a given pixel or set of pixels, image data along the Z axismay be averaged to calculate the data for the composite image at thatpixel or set of pixels.

In some embodiments, the composite image may be a two-dimensional imageor a three-dimensional image illustrating the depths associated with thetarget area. In some embodiments, two or more composite images may begenerated, each derived at least in part from all or a subset the imagedata associated with the plurality of focal planes.

FIG. 6 illustrates an example process 600 for generating a compositeimage of a target area with an extended depth of field, in accordancewith embodiments. Process 600 may be implemented as part of step 506 ofprocess 500 discussed above in connection with FIG. 5. Aspects of theprocess 600 may be performed, for example, by the image processing anddisplay system 106 discussed in connection with FIG. 1.

In an embodiment, process 600 includes generating 602 a plurality ofimages respectively corresponding to the plurality of focal planes. Insome embodiments, generating 602 the plurality of images includesscanning a target area on a plurality of focal planes, simultaneously orsequentially, individually or as a whole, collecting and detecting thebackscattered light as a result of the scan, and generating imagescorresponding to each of the plurality of focal planes. In otherembodiments, rather than generating a separate image for each of thefocal planes, a collection of image data representing some or all of thefocal planes may be generated and analyzed.

In an embodiment, process 600 includes identifying 604, for each of theplurality of images, a high-resolution portion of the image. Forexample, image data associated with each pixel or a collection of pixelsmay be analyzed. In some embodiments, given the relative positions andorientation of the scanning optical fiber tips, certain characteristicsassociated with each of the focal planes, such as the location of thehighest-resolution portion of the image, may be pre-calculated.

In an embodiment, process 600 includes combining 606 the high-resolutionportions of the plurality of images to form the composite image. Variousimage processing techniques, such as image stitching techniques, may beused. In some embodiments, rather than selecting only thehighest-resolution portions, the image data associated with images ofvarious depths of focus or field may be combined in other ways (e.g.,averaged).

In various embodiments, methods, systems and/or components describedherein may be used for image acquisition, image display/projection orboth. For example, the techniques described herein may be used in ascanning laser display system such as described in U.S. patentapplication Ser. No. 12/468,832, entitled “Scanning laser projectiondisplay devices and methods for projecting one or more images onto asurface with a light-scanning optical fiber,” filed on May 19, 2009, thedisclosure of which is incorporated herein by reference. In other words,the image acquisition techniques using staggered fiber tips describedherein can be used in reverse, as image display techniques. In such adisplay system, the light scanned across a target area is modulated overtime, such that image elements are rendered in the target area. Byproviding different image data via different fibers or other lightsources that are scanned across space, multiple display focal planes canbe generated. When viewed by the eye, these multiple focal planes may beviewed as a volumetric or stereo 3D image.

To this end, a 3-D scene or object may be sliced, by one or moreprocessors, into a plurality of discrete focal planes or layers. Each ofthe focal planes may be associated with a limited depth of focus ordepth of field but in combination, the focal planes may cover a fullrange of focus for the 3-D scene or object. In an embodiment, an opticalfiber assembly such as disclosed herein may be articulated in a scanpattern as discussed herein. When articulated, each optical fiber in theoptical fiber assembly may be configured to display an imagerepresentation associated with one of plurality of focal planesdiscussed above. In various embodiments, image representationsassociated with the plurality of discrete focal planes may be displayedsimultaneously or sequentially onto a target surface such as an eye.

Referring now to FIG. 1, the image scanning system 100 may be configuredto provide functionalities related to image acquisition, imageprojection, or both. For example, in an embodiment, the image scanningsystem 100 may be used as a projection system for projecting ordisplaying one or more images onto an area. The projection system may beused in a wide range of applications, for example, mobile phones,laptops, personal digital assistants (PDAs), MP3 players, smart phones,digital cameras, camcorders, personal mobile televisions, portableprojection units, GPS enabled devices, and automobiles.

In an embodiment, the processing and display system 106 includes amemory for storing a 3-D model of an object or a scene to be projectedonto an area. The 3-D model may be computer-generated. The processingand display system 106 may also include one or more processorsconfigured to generate a plurality of image representations based on the3-D model. For example, 3-D model may be “sliced” into a plurality of2-D images corresponding to distinct focal planes (e.g., along anoptical or Z axis). In various embodiments, each focal plane may beassociated with one or more image representations or sets of image data.Such images or image data may be stored in the memory. Based at least inpart on the image data, the processing and display system 106 maygenerate control signals to the optical scanner driver 108 to controlthe scanner actuator in the optical scanner assembly 118 as well ascontrol signals to the light source driver 114. In an embodiment, eachoptical fiber is controlled by a separate light source driver. In suchan embodiment, the signals received by a light source driver may begenerated in response to image data for one of the focal planes. Inanother embodiment, two or more optical fibers may share the same lightsource driver.

In an embodiment, the detectors 110 may be omitted from the projectionsystem. In another embodiment, the detectors 110 may be used to detectand/or measure light reflected from the projection area 101 to generatefeedback signals concerning the projected images. The feedback signalsmay be processed by the processing and display system 106. For example,by correlating where the projected light is in the scan pattern with thereflected light detected/measured, information relating to the image andany interactions with the image can be determined.

Referring now to FIG. 3, the scanning device 300 may be used for imageprojection purposes instead of or in addition to image acquisitionpurposes, in accordance with an embodiment. In this example, the opticalfiber assembly 301 is coupled to the scanner actuator 304, whichactuates the cantilevered distal portion of the optical fiber assemblyto scan in a scan pattern. Light scanned across a target area ismodulated over time, such that image elements are rendered in the targetarea. By providing different image data via different fibers or otherlight sources (such as provided by a multi-channeled cantilever assemblydiscussed in connection with FIG. 10) that are scanned across space,multiple display focal planes such as 324, 326, 328 and 340 can begenerated. When viewed by the eye, these multiple focal planes may beviewed as a volumetric or stereo 3D image.

FIG. 11 illustrates an example process 1100 for projecting a compositeimage onto an area, in accordance with embodiments. Aspects of theprocess 1100 may be performed, for example, by the image scanning system100 discussed in connection with FIG. 1. In an embodiment, the compositeimage may include a 3-D or stereo image comprising one or more 2-Dimages.

In an embodiment, process 1100 includes scanning 1102 an optical fiberassembly such as discussed herein in a scan pattern so as to provideillumination to an area, the optical fiber assembly comprising aplurality of optical fibers optically coupled to an optical system tofocus light from the optical fiber assembly and produce a plurality offocal planes located at different distances with respect to an opticalaxis in the optical system. In an embodiment, a 3-D composite image or3-D model of a scene or object may be generated or obtained. The 3-Dcomposite image or model may be processed to determine a plurality of2-D images associated with different focal planes. Control signals maybe determined from the optical scanner driver and/or light sourcedriver(s) discussed above. Such control signals may be used toarticulate the optical fiber assembly so as to generate the plurality of2-D images for the 3-D composite image or model. In some embodiments,the composite image may be generated based at least in part on the lightprovided to the target area. In various embodiments, the optical fiberassembly may be configured to project the plurality of 2-D images insimultaneously or sequentially.

In an embodiment, process 1100 includes modulating 1104 the intensity ofthe provided light to create variation in light intensity across thetarget area. In many embodiments, light source intensity, image size,scanning pattern, and/or scanning parameters are varied and/or modifiedto achieve desired image characteristics. Such variations and/ormodifications can be used to achieve desired image characteristics for avariety of target surface characteristics (e.g., size, orientation, andreflectivity) and ambient conditions (e.g., illumination). In anembodiment, the modulation of the light intensities is based at least inpart on the image or images to be projected.

In an embodiment, process 1100 includes projecting 1106, based at leastin part on the light provided to the target area, a composite imagehaving a depth of field spanning over a distance between the pluralityof focal planes. As discussed above, in an embodiment, the compositeimage is generated based at least in part on a 3-D model or imagecontained within a processing and display system such as discussed inconnection with FIG. 1. In an embodiment, the composite image may begenerated by superimposing the plurality of 2-D images projected by theplurality of optical fibers in the optical fiber assembly. Each of thefocal planes may be associated with a limited depth of focus or depth offield but in combination, the focal planes may cover a full range offocus for the 3-D scene or object.

In an embodiment, detectors may be used to detect and/or measure lightreflected from the projection area to generate feedback signalsconcerning the projected images. The feedback signals may be processedby the processing and display system 100. For example, by correlatingwhere the projected light is in the scan pattern with the reflectedlight detected/measured, information relating to the image and anyinteractions with the image can be determined.

FIG. 7 illustrates some example configurations of optical fibers in anoptical fiber assembly, in accordance with some embodiments. In variousembodiments, the number, dimension, relative positions and othercharacteristics of the optical fibers may be configured as desired basedat least in part on various considerations such as the application ofthe optical fiber assembly and/or scanning device. In some embodiments,the optical fibers may be arranged side by side as in configurations702, 704 and 706 such that the axes of all the optical fibers are in oneplane. In other embodiments, the optical fibers may be arranged such asshown in configuration 708 such that at least some of the axes are indifferent planes.

In an embodiment, such as shown in configuration 702, the tips ofadjacent optical fibers are staggered along one direction (e.g., towardthe proximal or distal direction) from an outermost optical fiber towardthe other outermost optical fiber. In another embodiment, such as shownin configuration 704, the tips of adjacent optical fibers are staggeredin more than one direction (e.g., toward the proximal and the distaldirections) from one outermost optical fiber toward the other outermostoptical fiber. In yet another embodiment, such as shown in configuration706, at least two of the optical fibers are flush (i.e., not staggered)at the tips.

While four optical fibers are shown here to illustrate certain concepts,it is understood that in various embodiments, the optical fiber assemblymay include more or less than four optical fibers. In addition, theoptical fibers may have the same or different cross-section profiles,materials and the like. For example, one of the optical fibers may havea thicker core and/or cladding than another optical fiber. For anotherexample, one of the optical fibers may be a multi-mode optical fiberwhile the others may be single-mode optical fibers. In some embodiments,the claddings of at least some of the optical fibers may be etched orotherwise tapered at the distal portions to decrease the overalldiameter of the optical fibers. The numerical aperture of the lightemitting from the optical fiber or waveguide may be different due to theinherent structure of the waveguide (e.g. refractive indices) or due tomicrolens on the distal end of the waveguide.

In various embodiments, the end faces or tips of the optical fibers inthe optical fiber assembly may be fabricated with a geometry and/ordesign to reduce light loss or back reflection. For example, in anembodiment, the axially staggered tips of the optical fibers is encasedin a refractive index matching medium so there is little or no air-glasssurfaces that would scatter light before light emerges from a single endface. The material for the refractive index matching medium may includean index-matching optical epoxy. In an embodiment, each optical fiber isfused to a strand of glass rod (not an optical fiber with core andcladding materials), then etched, and the optical fiber material isstaggered while the glass rod material is cut at the same point for allwaveguides to be fused together.

In some embodiments, individual end faces of staggered optical fibersmay be cut at a small angle so that the emerging cone of light from theoptical fibers do not strike adjacent optical fiber that may extendbeyond the shorter optical fiber. FIG. 8 illustrates some examplesoptical fibers in an optical fiber assembly with angled end faces, inaccordance with some embodiments. For example, as shown, the end facesof optical fibers in optical fiber assembly 802A are cut at an anglerelative to an axis perpendicular to the axis of the optical fibercores, such that the resulting end faces form a smooth slanted opticalsurface. In various embodiments, the exact angle to be cut may depend onthe staggered distance between optical fibers, refraction index of theend face materials, characteristics of the optical system used to focusthe light, location of the target area and other considerations. Across-section view of the optical fiber assembly 802A is provided as802B. As shown, light comes out of such an angled end face at an anglerelative to the axis of the fiber, for example, due to the refractiveindex of the end face material.

Advantageously, cutting the end faces of the optical fibers according toconfiguration 802A and B may facilitate simplified manufacturing ofstaggered optical fiber tips discussed herein. Specifically, rather thancutting (at an angle or not) and polishing the end face of each opticalfibers individually and binding them in a staggered fashion, the opticalfibers may be simply bound together in a bundle without regard towhether the fiber tips are aligned or staggered. Thereafter, the distalportion of the combined optical fiber bundle may be cut and polished asa whole, rather than individually, to produce a smooth slanted opticalsurface.

A different end face configuration for the optical fibers in an opticalfiber assembly is shown by 804A and B (cross-section view). As shown,the end faces of the optical fibers are cut at an angle relative to theaxis perpendicular to the axis of the optical fiber cores. However, theangled end faces are shown as facing an opposite direction as that shownin configuration 802A and B.

A close-up view of an angled end face of an optical fiber 806 isprovided. As shown, the end face of the optical fiber may be cut at anangle θ₁ relative to an axis 810 perpendicular to the axis 808 of theoptical fiber core. The resulting light or light cone coming out of theoptical fiber tip may deviate from the axis 808 of the optical fibercore at an angle θ₂. In various embodiments, the end face angle θ₁and/or the end face material may be adjusted to vary the resulting lightcone angle θ₂.

FIGS. 9a-c illustrate an example optical fiber assembly 900, inaccordance with an embodiment. In this example, the optical fiberassembly 900 includes a bundle (e.g., seven) of optical fibers withaxially staggered distal ends, where one optical fiber is in the centerof the bundle surrounded by the rest of the optical fibers. In someembodiments, the center optical fiber may or may not have the longestdistal length such that the distal end of the center optical is theclosest to the optical system. In some embodiments, at least some of theoptical fibers may have the same distal length so that their distal tipsare flush at the end face. While FIGS. 9a-c show seven optical fibers inthe optical fiber assembly, it is understood that in various embodimentmore or less optical fibers than shown may be included in the opticalfiber assembly.

FIG. 9a illustrates a cross-sectional view showing three of the sevenoptical fibers of the optical fiber assembly 900. As shown, the centerfiber 902 has an un-angled end face that is perpendicular to the coreaxis of the optical fiber whereas the surrounding fibers 904 and 906 mayhave angled end faces facing away from the center optical fiber. In thisway, the light beams emerging from the distal ends of the optical fibersmay overlap off the distal end for the application to form a distalplane where all optical beams intersect distally from the tip along anaxis of the optical fiber assembly 900. The end faces of the opticalfibers 904 and 906 may be cut at the same or different angles. Invarious embodiments, the characteristics of the emerging optical beamsmay vary due to the different (e.g., staggered) lengths of the distalportions of the optical fibers. In some embodiments, different numericalapertures may be applied to the optical fibers to compensate for suchdifferent characteristics resulting from the staggered distal lengths.

FIG. 9b illustrates a side view showing five of the seven optical fibersof the optical fiber assembly 900. As shown, the distal ends of thecores of the optical fibers 902, 904, 906, 908 and 910 may be located atdifferent locations (e.g., x5, x4, x3, x2 and x1, respectively) along alongitudinal axis 920 of the optical fiber assembly 900.

FIG. 9c illustrates an end view showing the seven end faces of theoptical fiber assembly 900. As shown, the optical fiber assembly 900comprises the center optical fiber 902 surrounded by six optical fibers904, 906, 908, 910, 912 and 914.

In some embodiments, the angle of the tip of the optical fiber can bedifferent for at least some of the optical fibers in an optical fiberassembly. For example a sharper angle could be used for the longer fiberor fiber core further from the bundle central axis, while a shallowerangle can be used for a shorter fiber or fiber core closer to the fiberbundle optical axis. The purpose may be to bring at least some of theangled beams together at the same optical beam size at the same planedistal to the optical fiber assembly.

In some embodiments, the overall diameter of an optical fiber could bedifferent for at least some of the optical fibers such that the cores ofthe optical fibers would be located at different lateral distances fromthe central axis of the optical fiber assembly.

In some embodiments, the numerical aperture for at least some of theoptical fibers could be different. The numeric apertures may determinethe size of the optical beam (cross-sectional diameter) over length fromthe tip of the optical fiber. Such different numeric apertures may beprovided by micro-lenses or other optical components located at the tipsof the optical fibers.

In some embodiments, instead of an optical fiber assembly, amulti-channeled cantilever assembly with staggered tips may be used. Thematerial of this mechanical cantilever can be fabricated from one ormore of a non-optically transmissive property, such as metal, ceramic,or plastic with the capacity (e.g. embedded wires) to bring electricalenergy to the distal tip where light is being generated, and optionallyto dissipate heat. The multi-channel cantilever may be driven (e.g., bya piezoelectric actuator) to scan in a scan pattern similar to the wayan optical fiber assembly is driven to scan. In an embodiment, themulti-channel cantilever may be driven at or near its mechanicalresonance for amplification of its lateral vibratory scan motion. FIGS.10a-d illustrate some example mechanical cantilever assemblies, inaccordance with some embodiments. FIG. 10a illustrates an end view of anexample multi-channeled cantilever assembly 1000. As shown, themulti-channeled cantilever assembly 1000 may include threelight-delivering members 1002, 1004 and 1005, each configured to provideillumination from respective distal light sources 1003, 1005 and 1007such as Light Emitting Diode (LED) or vertical cavity surface emittinglaser (VSCEL).

FIG. 10b illustrates a cross-sectional view of the multi-channeledcantilever assembly 1000, in accordance with an embodiment. The viewshows two of the three light-delivering members 1002 and 1004. In thisembodiment, each light-delivering member includes a light source at adistal end of the light-delivering member. For example, thelight-delivering members 1002 and 1004 include distal light sources 1003and 1005, respectively. Power or electrical signals to the light sourcesmay be provided externally by wires (e.g., wires 1001 and 1009) and/orby batteries coupled to the light sources. In some embodiments, thewires may be enclosed within the light-delivering members. In someembodiments, such as shown in FIG. 10b , the light delivering membersmay be arranged in an axially staggered manner such that the distallight sources are staggered along a longitude axis of the mechanicalcantilever.

In other embodiments, such as shown in a cross-sectional view in FIG.10c , the distal light sources may not be axially staggered along alongitude axis as shown in FIG. 10b . Rather, other mechanisms may beused to vary the characteristics associated with the light sources. Forexample, some of the light sources such as light sources 1010, 1012 and1014 may have one or more micro-lens or other optical componentsoptically coupled to the light sources (individually or as a group) suchthat the emerging light beams may be of different size, collimation,intensity and/or other characteristics. Some of the light sources maynot be coupled to any micro-lens or optical components at all.

FIG. 10d illustrates an end view of the mechanical cantilever shown inFIG. 10c . As shown, the light sources 1010, 1012 and 1014 may becoupled, respectively to micro-lenses 1011, 1013 and 1015. Themicro-lenses may be of different characteristics including powers. Forexample, the micro-lens 1011, 1015 and 1013 may be of increasing power.In various embodiments, the use of micro-lens and/or the types ofmicro-lens to use may depend on the application of the mechanicalcantilever.

In various embodiments, any light source may be used as distal lightsources including LED, VSCEL and the like. For example, the distal lightsources in FIG. 10b may be LED sources whereas the distal light sourcesin FIG. 10c may be VSCEL sources. In some embodiments, a singlemechanical cantilever may include light sources of the same or differenttypes.

The example embodiments described above is only representative of manydifferent ways of projecting or acquiring multi-focal imagessimultaneously or sequentially. In additional embodiments, multiplescanning fiber and optical systems may be employed in parallel toacquire or project multiple images at high resolution and short focaldepth, while in combination these images create a single image ormultiple composite or stereo images of high resolution and extendeddepth(s) of focus or field.

An example is provided below to illustrate the methods and techniquesdescribed herein. In particular, this example illustrates means tocreate an extended depth of focus while maintaining high imageresolution in scanning fiber imaging system, such as a scanning fiberendoscope (SFE). Typically, an SFE has a single mode optical fiber thatis driven in a spiral pattern to deliver illumination in a space-fillinground area over the surface to be imaged. The optical system (objectivelens system) between the scanning fiber tip and the tissue defines theimage resolution in the SFE. The optical system is typically designedfor high spatial resolution in the images, thus being having anundesirably short depth of focus or field.

To achieve extended depth of focus or field, the single optical fiberscanner is replaced with an opto-mechanical scanning cantilever that hasmore than one waveguide and more than one effective point source for theillumination. By varying the effective source point relative to theobjective lens in a fixed optical system, the focal point is shifted atthe illumination plane on the surface of a target area or object.

In this example, three or four single mode optical fibers made fromsilicon dioxide (quartz) material have the cladding etched from thestandard 125 microns in outer diameter to approximately 10 to 50 micronsin outer diameter, and in length from the free distal tip to about 2 to8 mm proximally. The three or four optical fibers are held together(side by side) and their tips trimmed to make an optical smooth end-facewith a carbon dioxide laser. The proximal ends of the optical fiberstrands are pulled so the tips are not all flush or end at the samepoint at their free ends. Each end-face of the optical fibers isstaggered by a small axial distance (e.g., less than 1 to 2 mm) tocreate separate and axially shifted or staggered illumination planesusing a single objective lens system.

Once the geometry of the three or four parallel optical fibers has beenestablished, the etched regions of the three or four fibers are bondedtogether using any one or a combination of the following techniques: (1)using carbon dioxide laser to weld the fibers together; (2) using heatto weld the fiber together; (3) using optical epoxy to bind the fiberstogether; (4) using adhesive or chemical to bind the fibers; and/or (5)using physical tubing that may be shrunk to hold the fibers together.The fused opto-mechanical system would be driven into a resonantvibratory mode in similar fashion as the SFE to produce a space-fillingillumination. However, this illumination will produce three or fourdifferent planes of focus that can give three or four different highresolution images if the backscatter light is detected in aframe-sequential, scan-line-sequential, or even pixel-sequential manner.In this case, the same set of red, green, or blue optical detectors maybe used for imaging from each of the three or four differentillumination planes to generate an extended depth of field,high-resolution image of a surface with topographic relief. In aconfocal arrangement, separate individual optical detectors of light(e.g. fluorescence) may be used to measure optical signals collectedwithin the core of each optical fiber that was used to illuminate aparticular focal plane.

Each illumination plane can have a very limited depth of focus or fieldof view, but in combination the resulting images of the target area canhave the desired extended depth of focus at the highest resolution. Forexample, each of the three or four two-dimensional images resulting fromthe embodiment above may be computer processed to identify the regionsor pixels within the image of highest focus. Knowing beforehand wherethe focal depth is for each illumination fiber, the regions (pixels) ofhighest focus may be combined in a three-dimensional mapping to thecorrect X and Y pixel location of the 2D image and the Z location ofeach of the three or four imaging planes. In some embodiments, thesharpest-focused portion of the composite image may come from only oneof the illumination planes for a given Z location.

In the example, the cladding of the cantilevered optical fibers does nothave to be etched first before being bound together. The etchingprocedure is proposed to simply save space for the microendoscopyapplication and create a multi-waveguide quartz cantilever ofapproximately the same overall diameter as our current 80 to 125 micronsingle optical fiber used within the micro-optical fiber scanner.

In the example, the end faces of the optical fibers may be cut andpolished at an angle such as illustrated in FIG. 8 so that lightemerging from an optical fiber tip does not strike an adjacent opticalfiber. The angle may be approximately 8 degrees relative to the axisperpendicular to the optical core. The resulting light cone emitted fromthe angled end face may deviate approximately 12 degrees from theoptical core. In various embodiments, the cut angle and the deviationangle may be more or less than 8 and 12 degrees, respectively.

While preferred embodiments of the present invention have been shown anddescribed herein, it will be obvious to those skilled in the art thatsuch embodiments are provided by way of example only. Numerousvariations, changes, and substitutions will now occur to those skilledin the art without departing from the invention. It should be understoodthat various alternatives to the embodiments of the invention describedherein may be employed in practicing the invention. It is intended thatthe following claims define the scope of the invention and that methodsand structures within the scope of these claims and their equivalents becovered thereby.

What is claimed is:
 1. A method for projecting a composite image to atarget area, comprising: scanning an optical fiber assembly in a scanpattern so as to provide illumination to the target area, the opticalfiber assembly comprising a plurality of optical fibers opticallycoupled to an optical system to focus light from the optical fiberassembly and produce a plurality of focal planes located at differentdistances with respect to an optical axis in the optical system;modulating the intensity of the provided light based on an image tocreate variation in light intensity across the target area; andprojecting, based at least in part on the illumination provided to thetarget area, a composite image based on the image onto the target area,the composite image having a depth of field spanning over a distancebetween the plurality of focal planes.
 2. The method of claim 1, whereinthe plurality of optical fibers are arranged in an axially staggeredfashion.
 3. The method of claim 1, wherein at least two of the pluralityof optical fibers are coupled to each other.
 4. The method of claim 1,wherein scanning the plurality of optical fibers includes actuating, bya scanner actuator, the distal portions of the plurality of opticalfibers in a scan pattern.
 5. The method of claim 4, wherein the opticalsystem remains substantially stationary relative to the optical fiberassembly.
 6. The method of claim 1, wherein projecting the compositeimage includes projecting a plurality of images respectivelycorresponding to the plurality of focal planes.
 7. The method of claim6, wherein each of the plurality of images are projected by one of theplurality of optical fibers.
 8. The method of claim 6, wherein theplurality of images are projected simultaneously.
 9. The method of claim6, wherein the composite image is three-dimensional and the plurality ofimages are two-dimensional.
 10. The method of claim 1, wherein at leastsome of the plurality of optical fibers have angled distal end-faces.11. A system for projecting images onto a target area, comprising: anoptical fiber assembly comprising a plurality of optical fibers operableto provide illumination to the target area; an optical system opticallycoupled with the optical fiber assembly, the optical system operable tofocus illumination from optical fiber assembly to produce a plurality offocal planes located at different distances with respect to an opticalaxis in the optical system; a scanner actuator coupled to the opticalfiber assembly configured to actuate distal portions associated with theplurality of optical fibers to scan, either together or individually, ina scan pattern on the plurality of focal planes; and one or moreprocessors comprising a tangible medium, the tangible medium comprisinginstructions that when executed cause the one or more processors toproject, based at least in part on the illumination provided to thetarget area, a 3-D composite image having a depth of field spanning overa distance between the plurality of focal planes.
 12. The system ofclaim 11, wherein the plurality of optical fibers are arranged in anaxially staggered fashion.
 13. The system of claim 11, wherein at leasttwo of the plurality of optical fibers are coupled to each other. 14.The system of claim 11, wherein scanning the plurality of optical fibersincludes actuating, by a scanner actuator, the distal portions of theplurality of optical fibers in a scan pattern.
 15. The system of claim14, wherein the optical system remains substantially stationary relativeto the optical fiber assembly.
 16. The system of claim 11, whereinprojecting the 3-D composite image includes projecting a plurality ofimages respectively corresponding to the plurality of focal planes. 17.The system of claim 16, wherein each of the plurality of images areprojected by one of the plurality of optical fibers.
 18. The system ofclaim 16, wherein the plurality of images are projected simultaneously.19. The system of claim 16, wherein the plurality of images aretwo-dimensional.
 20. The system of claim 11, wherein at least some ofthe plurality of optical fibers have angled distal end-faces.